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Abstract 

Let {Si}, ... , {S^} be the one-parametric groups of shifts along the orbits of 
Hamiltonian systems generated by time- independent Hamiltonians Hi , . . . , Hn with 
one degree of freedom. In some problems of population genetics there appear planar 
transformations having the form 5^" • • • under some conditions on Hamiltonians 
Hi, ... , Hn. In this paper we study asymptotical properties of trajectories of such 
transformations. We show that under classical non-degeneracy condition on the 
Hamiltonians the trajectories stay in the invariant annuli for generic combinations 
of lengths hi,..., hn, while for the special case hi + ■ ■ ■ + hn = there exists a 
trajectory escaping to infinity. 

Mathematics Subject Classification: 37J40, 37J15, 37M05; 
Keywords: KAM -Theory, Hamiltonian Systems 

1 Introduction. 

Let Hi{p,q), H2{p,q) be two time- independent Hamiltonians with one degree of freedom. Typ- 
ical examples can be Hi = \J [j> — "PxY + ('7 — and H2 = \/ {p — P2Y ~^ {1 ~ I2Y where pi, 
qii P2, 92 are some constants. Sometimes instead of symplectic coordinates {p,q) we shall write 
{x,y) or in complex notation z = x + iy. Denote by {Si}, {S2} the one-parametric groups of 
shifts along the trajectories of the first and second system. Ya.G. Sinai formulated a general 
question about asymptotic properties of transformations T^^i''*^) = 5'^2 . ghi ^ where hi, /i2 are 
fixed numbers. 

Similar transformations appear in some problems of population genetics (see [2]). In the 
example described above, the map T^^^'^'^^ moves each point {p,q) along the circle with center 
(pi , qi ) for the distance hi in the positive direction and then moves its image along the circle 
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centred at {p2,Q2) for the distance /12 (see Fig. [T]). For negative values of h^, we assume the 
movement in the opposite direction. 

One can ask a natural question about boundedness of the trajectories (T^^^'^^^") ip,Q)- In 
more detail: do invariant curves of the map T^^^''^^^ exist and separate the phase space on 
bounded annuli? From our results it follows that for /ii + /i2 7^ the answers to these questions 
are affirmative, while for /ii + /12 = the answers are negative. 

Assume that in the previous example Pi = qi = 0. Then Hi = \Jp^ + and the corre- 
sponding Hamiltonian flow has the form 

^t = ^.+ '- 
^Pt = Po 

in the usual polar coordinates 93 = arg(2;), p = \z\. For any t, this map is symplectic, i.e., it 
preserves the area element dx Ady = pdp A di^. 

Definition 1 For a given point F in the plane and the usual polar coordinates {p, if) with the 
center F introduce the inverse polar coordinates with the center F: (r = — , tp). We call the map 
T ^ 

^ ^ (2) 

n = r 

the skew rotation with the center F and with angular speed h. 

Definition 2 The map is called the perturbed skew rotation if it is invertible, symplectic and 
has the asymptotics 

(ifi = if + hr + 0{r^) 
In = r + 0(r 



(3) 



as r — )• 0. 

Note that in this definition ((/?, r) G [0, 27r) x M^. are not necessarily the inverse polar coor- 
dinates but any "polar" coordinate system in the neighbourhood of the infinity point r = 0. If 
the correction terms O(r^), O(r^) on the right-hand side of ([3]) are absent, we call the map T/^ 
a generalized skew rotation. 

Remark. As it is known (see for example ^) a continuous time Hamiltonian system in the 
plane with closed trajectories can be written in the canonical form 

^ = ^{I) m 
7=0 ^ ^ 

for particular variables ((/?, /) G [0, 2tt) x R+ which are called action-angle coordinates. The 
function U!{I) is called frequency or angular velocity. The action variable / represents, up to 
the numerical factor, the area surrounded by the level curves of Hamiltonian. Let the frequency 
uj{I) monotonously tend to as / — )• 00. So we can use {uj,ip) as "polar" coordinates in a 
neighbourhood of infinity. In this coordinates, a shift along the orbits of the flow Q is the 
generalized skew rotation. 
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Figure 1: Product of two skew rotations around the points Fi and F2 with hi, h2 > 0. 

Let Ti and T2 be perturbed skew rotations in two different coordinate systems (r, ip) and 
(f, If) in a neighbourhood of inhnity. What can one say about the existence of closed invariant 
curves for the map T = Ti • T2? 

As we shah show, the quahtative behaviour of the product of several skew rotations is similar 
to the behaviour of the product of the usual rotations of the plane. Consider motions of the 
plane given, in complex notation, as z ^ aiZ + 6j, |oj| = 1. The product of these motions is 

N N N 

z az + b, a = Y\_ai, b = Y\_bi + bjsi-i ajq^j 

i=l i=l i=0 j<i 

If o 7^ 1 then this map is a rotation and so the trajectories of this map lie on the circles with the 

center z = — in the fixed point of this rotation. We generalize this result to the skew rotations 
a 

and perturbed skew rotations. Our result shows that the typical trajectories lie on the closed 
invariant curves. The rest of the trajectories are contained in the annuli surrounded by these 
curves. This follows from the topological lemma. 

Lemma 1 Let A be the annulus bounded by two closed curves 70 and 71. Let T be a homeo- 
morphism of the domain B D A. IfT{'yi) = ^i, i = 0, 1, then T{A) = A. 

On the heuristic level, our arguments are the following. A rotation around any center on the 
projective plane can be considered also as a rotation around infinite point. Thus the product of 
two rotations with different centres is a small perturbation of a rotation around infinity. For such 
a perturbed system with some non-degeneracy conditions, one can apply Kolmogorov-Arnold- 
Moser theory and solve the described problem. 

However some work is needed to make this scheme rigorous. We use a theorem formulated 
in other notations in ^ §34]. 
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Theorem 1 (J. Moser) Let T be a real analytic invertible map defined in some "polar" coor- 
dinates {(p, r) £ [0, 2tt) X as 



ri = r + 0(r 



for some £ G |N. Suppose also that T satisfies the intersection property: the image of any closed 
curve surrounding the point r = and sufficiently close to the origin intersects the original 
curve. 

Then T has an invariant curve surrounding the stable point r = in any neighbourhood of 
this point. 

The measure dipdr of all points that do not belong to the closed invariant curves of T is o(e) 
in the e -neighbourhood of the stable point r = 0. 

Theorem [1] with 21 = 1 can be applied to the perturbed skew rotations, due to the following 
lemma. 

Lemma 2 Any perturbed skew rotation T satisfies the intersection property. 

From the Lemma [2] and Theorem [H it follows that the perturbed skew rotation has infinitely 
many closed invariant curves in any neighbourhood of infinity. By Lemma [1] it means that the 
trajectory of any point is contained in some invariant annulus for the perturbed skew rotation. 

In the next section we present the proof of the following theorem. 

Theorem 2 Let Rj^hj denote skew rotations with centres Fj, j = 1,...,N. Then for any 

N N 

set [hi, ... ^h^) such that ^ hj ^ 0, the trajectories of the map T = Y\ Rj h c-i^s bounded. 
Actually, they are contained in invariant annuli (as it is mentioned above). 

N 

The case Yl ~ ^ considered in section [3] (for N = 2). In section H] we present natural 

generalisations of Theorem [2j In section [5] we analyze the non-smooth case. Finally, in section 
[6] we present the results of numerical computations. 
Proofs of Lemmas [2] and [1] are given in Appendix. 

Acknowledgements The authors thank Ya.G. Sinai for many fruitful discussions and for 
his substantial help in writing of this article. We are also grateful to M. Blank for useful remarks. 



2 Proof of Theorem [2 



Since the product of two perturbed skew rotations in the same coordinate system is again a 
perturbed skew rotation, to prove Theorem [2] it is sufficient to show that the skew rotation with 
one center is the perturbed skew rotation in the inverse polar coordinate system with any other 
center. We verify the last statement by direct calculation. 

Without loss of generality consider the case of the skew rotation with the center i*o = in 
the inverse polar coordinate system with the center Fi = 1. 
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The skew rotation -Ro,/io shift along the trajectories of the system 

(^=: (6) 

I r = 

where (97, r) are inverse polar coordinates with the center Fq. In the complex notation 99 = arg(z), 
1 

r = — . Inverse polar coordinates {(p,r) with the center Fi can be expressed as 
\z\ 



(f = arg(z — 1) = Im (ln(2; — 1)) = Im ( In 2; + In ( 1 



z 



Im ( In 2; h 0{z ^) ] = ip + r sin 99 + 0{r'^^ 



z 



(7) 



exp (-Re(ln(z - 1))) = exp ( -Re(lnz) - Re ( In ( 1 - i 



\z-l\ 

= 1^ (^1 + Re + 0(z"^) J = r(l + r cos 99) + ©(r^) 

Thus system ^ in coordinates (95, f) assumes the form 

{(^ = r(l + r cos 99 + 0(r^)) = f + O(r^) 
f = (— sin 99 + 0(r^))r = O(r^) 



(8) 



(9) 



Prom ([5]) it follows that 1 as r — )■ 0, which gives O(r^) = O(f^) in the neighbourhood 

r 

of r = 0. Since the map i?o,/io is a shift along the trajectories of ([6]) it is also a perturbed skew 
rotation in the coordinate system {(p,r). ■ 



3 The unbounded case. 

Here we analyze the case of products of two opposite skew rotations. 

Recall the notion of the composition of two Hamiltonian systems with Hamiltonians Hi and 
H2 as a system corresponding to the vector field which is equal to a linear combination of vector 
fields given by Hj, j = 1,2 (see ^). Such a system also can be written in Hamiltonian form 
with the Hamiltonian 

H = hiHi + h2H2. (10) 

In the case of our basic example ([T]) for hi 7^ — /i2; the level curves of the Hamiltonian H are 
closed and are called Cartesian Ovals - the algebraic curves which are defined by the equation 

hi\z — Fi\ + h2\z — F2I = const. (11) 

For the particular case hi = h2, the level curves of the Hamiltonian H is the family of confocal 
ellipses with foci Fi, F2. If /ii = — /i2 , the relation (jlip defines the family of confocal hyperbolas 
with the same foci. 

In this context Theorem [2] states that for /ii + /12 7^ the orbits of the map T^'^^''^^) {[q near 
the orbits of the completely integrable Hamiltonian system with Hamiltonian (llOp . For the case 
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Figure 2: Linear combination of Hamiltonians Hi and H2. 



hi = — /i2 the map T^^~^'^ is an 0(|z|~^) perturbation of the identity map and thus KAM - 
theory methods are not applicable. 

Let Fi = (—1,0) and = (1,0) be the two centres. Let Ri ^ and R2,-h be the skew 
rotations with centres Fi, F2 with opposite values of parameters. One has the following result. 

Theorem 3 The map T = Ri^h ■ -^2,-/1 has a trajectory escaping to infinity. 

Liformally, Theorem [3] states that, in the case hi = — /i2, the trajectories of the map T^^^'^^^ 
also follow trajectories of the Hamiltonian system with Hamiltonian (llOp . 

Proof. Consider an arc zq ^ zi of length h with center Fi starting at some point zq = xo + iyo, 
xq > 0, which intersects the imaginary axis in its middle point Zq = iy^. The distance from zi 
to the imaginary axis equals — Re(zi) = —xi = xq. The distance from Z2 = R2{zi) to the axis is 
greater than xq since Re(2;i) < and so \zi — F2I > \zq — F2I. Thus Re(z2) > Re(zo)- Similarly, 
for 2:3 = i2i(z2), we get Re(z3) < Re(zi). 

By the same argument the distance from the n-th image of the point zq to the imaginary axis 
is greater then that of the n — 1-st step. Incidentally, the arcs obtained on each iteration intersect 
the axis since \xj\ < h. Thanks to convexity of the circle, the sequence {Uj} of coordinates of 
the intersection of the arc Zj Zj+i with the line Re(z) = is increasing. Suppose, that this 
sequence has some finite limiting point y. Then for (0, y) one gets r(0, y) = (0, y) which is 
definitely impossible for any finite y. 

Similar considerations apply to any point z G C. The role of the axis x = in this case will 
be played by some hyperbola with foci Fj. Theorem [3] is proven. ■ 
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Figure 3: Hyperbolic trajectories. 

4 Generalisations. 

Now let us turn to the general case of a perturbed skew rotations given in a different coordinate 
systems. 

Definition 3 Two coordinate systems (r, ip) and (f, ip) are called concordant in the neighbour- 
hood of r = if 



1. The point r = 0, (/? G [0, 27r) corresponds to f = 0. So 



df 



0. 



'r=0 



2. The transformation (r, ip) — t- (f, Lp) is a real analytic invertible map. 

d(p 

3. This transformation asymptotically does not change the angles and radii, i.e., —— 

dip 



, dr 
and — 
or 



r=0 



1 



r=0 



Obviously the concordance of coordinate systems is an equivalence relation. 
Example. Inverse polar coordinate systems with different centres are concordant. 



Lemma 3 // coordinate systems (r, and (f, ip) are concordant in a neighbourhood of r = 
= 0{r'^) for r — )• 0. 



dr 

then — — 
dip 



r=0 



df 

Proof. From the definition 7— 

dip 



r=0 



^ d f df 
and — — 

dr \dip 



r=0 



d f df 

dp> \dr 



r=0 



Theorem 4 Let (r, ip) and (f, ip) be two coordinate systems concordant in a neighbourhood of 
r = 0. Let a real analytic invertible map T satisfy conditions ([3]) in coordinates {r,ip). Then T 
also satisfies conditions ([3]) in coordinates {f,<p). 
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Proof. First consider 

df df 
fi - f = — (ri - r) + -rr-i^l " V>)- 
or dip 

dv 

In a neighbourhood of r = 0, the function — is bounded and the factor (ri — r) is O(r^), due 

Or 

dv 

to conditions Q. For the second term, we get — — = O(r^) from LemmaOand i(pi —ip) = 0(r), 

dip 

due to conditions ([3]). So we have 

- f = 0(r3) = 0(f3) 

since f ^ r. 

For the difference of angular coordinates we have 

The first term is 0{r^) from the previous discussion, and the second term is 

^""-(^1 - (^) = (1 + 0(r))(/ir + 0(r2)) = /ir + ©(r^) = /if + 0(f2). 



dip 

Theorem [3] is proven. ■ 

Now we can generahze Theorem [2J 

Theorem 5 Let (r, ip) and (f, (p) he two coordinate systems concordant in a neighbourhood of 
r = 0. Let a real analytic invertible map Ti he a perturbed skew rotation in coordinates {r,ip), 
a real analytic invertible map T2 he a perturbed skew rotation in coordinates (f , ip) . Then the 
product T = T1T2 is a perturbed skew rotation in coordinates {r,ip) (or {f,ip) as well). If the 
sum of the "angular coefficients" does not equal to zero then the conclusion of Theorem{l\ holds 
for T. 

Proof. It fohows from Lemma [21 Theorem |4] and Theorem [H ■ 

Corollary 1 // there are two Hamiltonian systems with closed trajectories such that the func- 
tions oji, UJ2 monotonously tend to at infinity, the coordinate systems (931, cji) and ((^2)^2) are 
concordant and /ii + /12 7^ then for the product of hi - and /i2 -shifts along the trajectories of 
these Hamiltonian systems the conclusion of Theorem [7] holds. 

Remark. Theorem [5] can be easily generalized to the case of any finite collection of invertible 
maps Ti , . . . , T/v • 

5 Non-smooth case. 

In this section we show that the smoothness of the above discussed system is essential. Now we 
orient the plane opposite to the usual way. So the positive rotation direction is clockwise. Let 
Oi = (— 1,0) and O2 = (|,0). Recall the basic example of the product of two skew rotations 
with different centres. Transformation T moves each point (x, y) along the level curve of the 
first Hamiltonian for the distance h and then moves its image along the level curve of the second 
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Hamiltonian for the distance h. In the basic example such level curves form two one-parametric 
families of concentric circles with centres Oi and 02- Consider the analogous system with two 
families of concentric squares, i.e. Hi = \x — xi\ + \y — yi\ and H2 = \x — X2\ + \y — y2\- For 
this system we show the coexistence of countably many periodic orbits and countably many 
trajectories escaping to infinity with speed 0{y/t). 

Let = {{x,y) : \x — Xj\ + \y — yj\ = R}, j = 1,2 be two families of concentric squares 
with centres Oi and O2 such that the corresponding sides of the squares are parallel and O1O2 
belongs to the common diagonal of the squares. The trajectory consists of odd and even steps. 
For even steps the point is shifted along the side of the square of the first family for the distance 
h and for odd steps - along the side of the square from the second family for the distance h 

in the same direction. The plane is splitted onto three parts (see Fig. [S]): half-planes x > —, 
1 1,1 

X < —— and the strip |x| < — . In the half-planes both dynamics coincide, while in the strip 
there exist a non-zero angle between the curves of first and second family. 



1 

t 


\ /\ /\ / 
\ / \ / \ / 

XXX 

/ \ / \ / \ 
/ \/ \/ \ 


y 

A, 

i t 


i. t % 

> 


Oi O2 





Figure 4: Three regions. 

Let h = a\pl. The shift along the square for the distance h sufficiently far from the 
vertices of the square corresponds to the shift for the distance a in each coordinate in the positive 
or negative direction, depending on the part of the square. 

Consider the trajectory of the point (— |, /iq). Denote by hn the ordinate of the point of n-th 
entry to the strip, by anV^ the length of the remaining part of the shift inside the strip and by 
an the number of the family modulo 2 (see Fig. [5]). For the starting point the corresponding 
triple is {hQ,a,^). Whenever > a, the values (/i„+i, a„_|_i, a„+i) can be obtained by the 
following recurrent relations. 

Denote by 7„ the number of complete steps that the trajectory spends in the strip after the 
n-th entry and by /3n\/2 the initial part of the next step belonging to the strip. We have 



In 



1 



(12) 



/3„ = 1 - On - a7r, 



I - ar. 



I - an 



I - ar. 



(13) 



where [x] and {x} denote the integer and the fractional parts of x. For hn+i it immediately 
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Figure 5: Dynamics in the strip. 



Compute the number of complete steps of the trajectory before the next entry to the strip. 

' 2\hn+i\ + Pn - a] 

1 nus we get 



It is equal to 



and 



On + 7n + 1 + 



2\h 



n+1 



mod 2 



«n+i = a 1 



2|/lra+l| + /3n 



(15) 
(16) 



1 



In order to simplify the analysis let us consider the case a = — , m G N 



m 



Proposition 1 // /iq 

0"n = 0, for all n G N. 



I, I ^ iN, £ ^ 1 and qq = a then hn = (ni, (n G h^, ^ 1 and 



Proof proceeds by induction. From a„-i = a and (I13p it follows that /3n,-i = 0. Thus hn in 
(|14p is equal to — or —hn-i ± a and so it is again of the form £„a where 2^„ G N. Since 

'2\hr, 



from (11611. it follows that ar. 



1 



a. 



Let a = , ao = a and /iq be as above. Then it follows from (I12p that 7„ = 2m 

2m 



1 and 



-/i^. So after 4(^ + m) steps, the point (— ^, /io) returns to the initial position. 
For a = r, = a it follows that 7„ = 2m — 2 and so /in+i = —hn — ( — l)°"a. So if 



2m - 1' 

/iQ = ^a, ^ G N then /i2n+2 = + 2a and the trajectory is expanding. If h^ = £a, £ & \ N 
then /i2n+2 = ^2n and the trajectory is periodic with the period 41 + 2m — 1. More generally, 



1 



if a = — , A; G N, ao = a — e, e G [0, o) then for ho = £a + e, i G \ N the trajectory is again 
periodic. This complete the proof of coexistence of periodic and escaping orbits. 

Numerical experiments shows that for initial triples {ho, a, a) with generic ho and a the 
behaviour of the trajectory resembles a random walk (see Fig. [8|). 
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6 Numerical results. 



First we present the results showing that for the case of two rotations the arguments from the 
section [3] actually apply. Computer simulations show that a trajectory of the map T^^'^'^^^ at 
each step intersects one level curve of the Hamiltonian (jlOp . 




Figure 6: Left: Trajectory of the point z = 3 + 5i under the map 

Tif'iM) for hi = 2.5, 

/i2 = —3 intersects Cartesian Oval 5|2; — 1|— 6|z + l|= const (black dotted line depicts 
Cartesian Oval, upper grey curve consists of the points T"'{Si'^z), lower curve corresponds 
to T"(S'2^2;)). Right: Trajectory of the point z = 1.6 + Oi under the map T'^^^'^'^^ for 
hi = 2.5, /i2 = 0.25 intersects Cartesian Oval |2; — 1| + 10\z + 1| = const. 

The map r'-'^^''^^) shows a typical KAM-theory behaviour: the islands of stability surrounded 
by closed invariant curves. 




Figure 7: Trajectories of the map 

rpihiM) for ^ ^ 3 3 g^n^ j^j^jg^i points (0,2.419) 
(large ovals) and (0,3) (islands surrounding the periodic points). 
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For the non-smooth case the dynamics looks Uke random walk. Typical trajectory is pre- 
sented on figure [8l 




Figure 8: Left: The points of the trajectory. Right: The distance of the trajectory 
\T'^{x,y)\ from for the step h = 2.43a/2 and randomly choosen initial point (xq, yo): 
l^^ol + \yo\ = 1- 



Appendix. 

Proof of Lemma [2]. The perturbed skew rotation T is by definition an invertible area- 
preserving map. Consider any closed curve 7 surrounding points zi = and Z2 = T"^(0). 
Then T{'y) is a closed curve surrounding point zi = and so, due to the area-preserving prop- 
erty, T(7) n7 7^ 0. Since any closed curve surrounding the infinity in some of its neighbourhood 
also surrounds points zi and Z2, the intersection property in the neighbourhood of infinity holds. 
■ 

Proof of Lemma [1], Suppose that for some xq £ A we have T(xo) ^ A. Consider any curve 
7(t) C A having the following properties: 

1. 7(0) E 70, 7(1) e 71, 

2. yt G (0, 1) : 7(i) n (71 U 70) = 0, 

3. 3to G (0,1) : 7(to) =xo. 

Since T is a homeomorphism and T(7(to)) ^ A, the image T(7(0, 1)) intersects the boundary 
71 U 7o in some point 

yo = T{j{ti)) 

Then 7(^1) = r~^(yo) G 7((0, 1)) H (71 U 70) which contradicts with our choice of 7 (property 
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